Helping us to help you

I want us to be able to provide meaningful and timely help with your projects when other avenues have failed to unstick things.

There are a few things you can do to make it easier for us to help you.

Try other avenues first!

  • Try debugging your code with print statements or logging (see this chapter of the DES book for an intro!)
  • Make sure you’ve saved any changes!
  • Close your terminal and rerun (or, if using a notebook, hit ‘clear all outputs’ and ‘restart’)
  • Take a general look on Google
  • Take a look at StackOverflow
  • Ask the community via your project channel
  • Try using an AI tool (Perplexity and ChatGPT are both good options), but take everything it says with a pinch of salt…
    • make sure you go through code it provides carefully so you understand it, and consider writing (formal or informal) tests to verify it works properly

Make your code available…

If all else fails - I really am happy to help (and have taken a look at various issues for people this year)!

It helps if you can give me as much information as possible about

  • what you’ve already tried
  • what seems to be going wrong
  • what you’re trying to achieve, and why

Having your code available on GitHub is the easiest way for us to troubleshoot issues.

If you cannot provide it on GitHub, sending code files to us on Slack is acceptable, but please attach it as a file to a Slack message rather than copying and pasting.

… and runnable!

Please either provide

  • an environment.yml or requirements.txt (if you have made your own environment from scratch)
  • or details of which of the HSMA environments you have used, and any packages you’ve added into those environments

If you are using branches, please make it clear which branch we need to be looking at!

What if I’m not allowed to share?

We might still be able to help!

Can you give us the gist of your problem? Like before, this might involve talking about

  • what you’ve already tried
  • what seems to be going wrong
  • what you’re trying to achieve, and why

You could also look into

  • whether you’re able to share part or all of the code without data
  • providing simple fake data (which you could even make in something like Excel)
    • this is also a great option for wider sharing and reproducibility of your code

Making your code readable

Understanding someone else’s code can be tricky - and time-consuming - at the best of times.

The following will really help us!

  • Comments!
    • Extensive comments are our friends at this stage. Why have you taken a particular approach? What is a given function doing?
    • Too many comments > too few
  • Good variable and function names increase readability
    • e.g. rather than ‘x, y, z’ or ‘input_1, input_2’, make your variables more descriptive, like ‘total_referrals’, ‘number_of_nurses’
  • Commenting out (or removing) unused code
    • The shortcut of highlighting code and pressing CTRL + / can make it easier to get temporarily comment out bits of code
  • Adhering to PEP8 guidance on line length makes a big difference too

Making your code super readable with docstrings

One of the best ways to make your code even easier for us to work with is to use docstrings.

Docstrings are like a super-comment that makes it easier to understand your functions, classes, and methods - wherever you are accessing them in the code.

When you start using a function from a package and it pops up with a load of extra useful details, that’s its docstring you are seeing.

Writing docstrings

Working asynchronously

A ‘quick meeting’ can often take a lot more time than you’d think!

(and in all honesty, unless it’s something like a typo, I’m unlikely to be able to resolve it within a call - though if you’d strongly prefer this, I can work with it)

However, I can often squeeze in some time looking at problems and then send you details of

  • what I fixed
  • how I went about working out what to do!

Advanced GitHub Features

GitHub isn’t just a great place to version control your code…

It’s a great place to

  • plan your project
  • track problems
  • and get help too!

Tip

Want a reminder of how to use GitHub?

Take a look at session 7a here:

https://hsma.co.uk/hsma_content/ modules/current_module_details/ 7_git_and_web_development

Or scan the QR code

GitHub Issues

GitHub issues are a great built-in (and free) way of tracking aspects of your project.

They are almost a built in to-do list to help you track what needs to happen - and what’s already happened.

They can be particularly useful for collaborative projects, but can also be very handy for solo projects.

What can we use issues for?

The great thing about your repo… is that it’s your repo!

Established projects may have more rules about what can be an issue and how it’s formatted, but you can set the rules on your project.

So you could use issues for

  • Features you want to add
  • Data you need to obtain
  • Documentation you need to write
  • Minor (and major!) bugs you need to fix

Examples of projects using issues - vidigi

In the vidigi issues, I’m primarily talking to myself and using it as a big to-do list.

Examples of projects using issues

In another project, we have used it for three of us to be able to communicate asynchronously about the project

Examples of projects using issues - Helena’s DES

Helena has been using these very effectively in her project

https://github.com/Countess-of-Chester-Hospital-NHS-FT/Non-Elective-Flow-Simulation/issues?q=is%3Aissue%20

Communication via Issues - a sneak peak

Including to summon me! (more on this later)

The Issues Tab

In your repositories, whether they are public or private, you should see an ‘Issues’ tab.

Our First Issue

When we first load this up, there’s nothing here… so let’s change that!

We’ll click the ‘New Issue’ button

Populating our first issue

The minimum we need to provide is a title (though it’s a good idea to put in some description too)

Viewing our first issue

This will take us to a view where we can add comments to the issue.

  • We can edit the title with the ‘edit’ button
  • We can edit the description with the three dots
  • We can close the issue when we’ve resolved it

Viewing our issue list

Heading back to our ‘issues’ page we can now see this in our list of issues.

Labels

Labels allow us to

  • group issues
  • make it easy to quickly identify what an issue is

Github provides us with a selection of predefined labels that we can choose from.

Adding our own labels

By clicking on ‘edit labels’ we are taken to a page where we can create any label we fancy

Filtering by Labels

On our issues page, we can then click on the ‘labels’ dropdown to only find certain kinds of issue.

Warning

Note - the filter won’t happen until you click elsewhere on the screen (outside of the label popup)

Mentioning issues elsewhere

It can be really helpful to mention issues on other issues, or in pull requests!

Typing a # symbol brings up a list of recent issues, and by starting to type you can filter it down.

How mentioned issues display

Mentioned issues will turn into a link when you save the description or comment

And they’ll show up in the linked issues and pull requests too

Tagging People in issues

If you mention someone by their GitHub username - prefacing it with an @ symbol - then they will get a notification.

On a public repo - or one where you’ve added me as a collaborator - you could summon me like that!

Closing Issues

When you are done with an issue, we can write a comment (optional, but recommended!) and close it down.

Note

If we want to, we can mention the ID of the pull request that closes this issue, or even the exact commit hash!

Fancy issue closing

In pull requests, if we mention an issue by ID with the # notation, it will

  • expand the issue ID into the issue name
  • automatically close the issue when the pull request is merged in if you use certain keywords like ‘resolves’

Milestones

In your projects, you may have different milestones you are working towards.

You might want to keep track of the issues you are aiming to achieve by different stages of the project - and then track your progress towards that.

In vidigi, I just use it to track issues I’m planning to fix in each minor version

Milestones - another approach

In another project, we’ve tied it in more detail to due dates and goals.

Creating a milestone

From the issue page, click ‘milestones’.

Then ‘new milestone’

Populating milestone details

From this, you can then enter a title of your choice, an optional due date, and any details to flesh out things like key goals or requirements.

Adding issues to a milestone via the milestones page

Now, when we click on the milestone header, we are given the option to add any issue to our new milestone.

Projects

Projects are an optional extra that you can use to enhance your use of issues.

We can access them from the ‘Projects’ header.

Creating a new project

We’ll click ‘new project’

Choosing aa project template

We’ll be provided with a list of possible project layouts.

My favourite is Kanban - but feel free to experiment!

Setting the project name

Give your project a name.

Our project

Initially we’re presented with a list of empty columns.

We then click on ‘add item’

Adding issues to columns

This brings up a search bar

Clicking on the plus gives us some more options.

If we click ‘add items from repository’ we can access our current repo’s issues - or issues from other repos!

Managing your proejct

Once this is done, we have an issue in the column.

We can move this around as we progress. Moving it to done automatically closes it!

Managing columns

Clicking on the three dots allows us to change the default limit for issues per column (0 = no limit)

We can also add, delete and reorder columns (by clicking and dragging them).

Project Documentation and Validation

An Introduction to Project Documentation and its role in Model Validation

Why do I bang on so much about documentation?

In my experience, lack of documentation

  • hinders reuse and adaptation - by you as well as by others!
  • can mask serious issues with assumptions or methods

And the process of writing documentation is crucial for

  • helping you slow down and properly check your code is doing what you think it is
  • helping you be really confident in explaining your code to others

Quarto

Checklists

There are a wide range of checklists out there for

STRESS-DES

STRESS-DES is 20-item checklist. It was designed to improve the reporting of DES models, ensuring that your report includes enough information for others to understand, evaluate and replicate the findings of your work.

While it’s designed for DES models, you may still find it provides some useful prompts for other kinds of models.

STRESS-DES: Objectives

Section/Subsection Item Recommendation
Purpose of the model 1.1 Explain the background and objectives for the model.
Model outputs 1.2 Define all quantitative performance measures that are reported, using equations where necessary.

Specify how and when they are calculated during the model run along with how any measures of error such as confidence intervals are calculated.
Experimentation aims 1.3 If the model has been used for experimentation, state the objectives that it was used to investigate.

(A) Scenario based analysis – Provide a name and description for each scenario, providing a rationale for the choice of scenarios and ensure that item 2.3 (below) is completed.

(B) Design of experiments – Provide details of the overall design of the experiments with reference to performance measures and their parameters (provide further details in data below).

(C) Simulation Optimisation – (if appropriate) Provide full details of what is to be optimised, the parameters that were included and the algorithm(s) that was be used. Where possible provide a citation of the algorithm(s).

STRESS-DES: Logic

Section/Subsection Item Recommendation
Base model overview diagram 2.1 Describe the base model using appropriate diagrams and description. This could include one or more process flow, activity cycle or equivalent diagrams sufficient to describe the model to readers. Avoid complicated diagrams in the main text.

The goal is to describe the breadth and depth of the model with respect to the system being studied.
Base model logic 2.2 Give details of the base model logic. Give additional model logic details sufficient to communicate to the reader how the model works..
Scenario logic 2.3 Give details of the logical difference between the base case model and scenarios (if any). This could be incorporated as text or where differences are substantial could be incorporated in the same manner as 2.2..
Algorithms 2.4 Provide further detail on any algorithms in the model that (for example) mimic complex or manual processes in the real world (i.e. scheduling of arrivals/ appointments/ operations/ maintenance, operation of a conveyor system, machine breakdowns, etc.).

Sufficient detail should be included (or referred to in other published work) for the algorithms to be reproducible. Pseudo-code may be used to describe an algorithm.

STRESS-DES: Logic (contd.)

Section/Subsection Item Recommendation
Components - entities 2.5.1 Give details of all entities within the simulation including a description of their role in the model and a description of all their attributes.
Components - activities 2.5.2 Describe the activities that entities engage in within the model. Provide details of entity routing into and out of the activity.
Components - resources 2.5.3 List all the resources included within the model and which activities make use of them.
Components - queues 2.5.4 Give details of the assumed queuing discipline used in the model (e.g. First in First Out, Last in First Out, prioritisation, etc.). Where one or more queues have a different discipline from the rest, provide a list of queues, indicating the queuing discipline used for each.

If reneging, balking or jockeying occur, etc., provide details of the rules. Detail any delays or capacity constraints on the queues.
Components - entry/exit points 2.5.5 Give details of the model boundaries i.e. all arrival and exit points of entities. Detail the arrival mechanism (e.g. ‘thinning’ to mimic a non-homogenous Poisson process or balking).

STRESS-DES: Data

Section/Subsection Item Recommendation
Data sources 3.1 List and detail all data sources. Sources may include:
• Interviews with stakeholders,
• Samples of routinely collected data,
• Prospectively collected samples for the purpose of the simulation study,
• Public domain data published in either academic or organisational literature. Provide, where possible, the link and DOI to the data or reference to published literature.

All data source descriptions should include details of the sample size, sample date ranges and use within the study.
Pre-processing 3.2 Provide details of any data manipulation that has taken place before its use in the simulation, e.g. interpolation to account for missing data or the removal of outliers.

STRESS-DES: Data (contd.)

Section/Subsection Item Recommendation
Input parameters 3.3 List all input variables in the model. Provide a description of their use and include parameter values. For stochastic inputs provide details of any continuous, discrete or empirical distributions used along with all associated parameters. Give details of all time dependent parameters and correlation.

Clearly state:
• Base case data
• Data use in experimentation, where different from the base case.
• Where optimisation or design of experiments has been used, state the range of values that parameters can take.
• Where theoretical distributions are used, state how these were selected and prioritised above other candidate distributions.
Assumptions 3.4 Where data or knowledge of the real system is unavailable what assumptions are included in the model?

This might include parameter values, distributions or routing logic within the model.

STRESS-DES: Experimentation

Section/Subsection Item Recommendation
Initialisation 4.1 Report if the system modelled is terminating or non-terminating. State if a warm-up period has been used, its length and the analysis method used to select it.

For terminating systems state the stopping condition.
State what if any initial model conditions have been included, e.g., pre-loaded queues and activities. Report whether initialisation of these variables is deterministic or stochastic.
Run length 4.2 Detail the run length of the simulation model and time units.
Estimation approach 4.3 State the method used to account for the stochasticity: For example, two common methods are multiple replications or batch means.

Where multiple replications have been used, state the number of replications and for batch means, indicate the batch length and whether the batch means procedure is standard, spaced or overlapping.

For both procedures provide a justification for the methods used and the number of replications/size of batches.

STRESS-DES: Implementation

Section/Subsection Item Recommendation
Software or programming language 5.1 State the operating system and version and build number.

State the name, version and build number of commercial or open source DES software that the model is implemented in.

State the name and version of general-purpose programming languages used (e.g. Python 3.5).

Where frameworks and libraries have been used provide all details including version numbers.
Random sampling 5.2 State the algorithm used to generate random samples in the software/programming language used e.g. Mersenne Twister.

If common random numbers are used, state how seeds (or random number streams) are distributed among sampling processes.

STRESS-DES: Implementation (contd.)

Section/Subsection Item Recommendation
Model execution 5.3 State the event processing mechanism used e.g. three phase, event, activity, process interaction.

Note that in some commercial software the event processing mechanism may not be published. In these cases authors should adhere to item 5.1 software recommendations.

State all priority rules included if entities/activities compete for resources.

If the model is parallel, distributed and/or use grid or cloud computing, etc., state and preferably reference the technology used. For parallel and distributed simulations the time management algorithms used. If the HLA is used then state the version of the standard, which run-time infrastructure (and version), and any supporting documents (FOMs, etc.)
System specification 5.4 State the model run time and specification of hardware used. This is particularly important for large scale models that require substantial computing power. For parallel, distributed and/or use grid or cloud computing, etc. state the details of all systems used in the implementation (processors, network, etc.).

STRESS-DES: Code access

Section/Subsection Item Recommendation
Computer model sharing statement 6.1 Describe how someone could obtain the model described in the paper, the simulation software and any other associated software (or hardware) needed to reproduce the results. Provide, where possible, the link and Digital Object Identifiers (DOIs) to these.

The STARS Framework

STARS stands for ‘Sharing Tools and Artefacts for Reusable Simulation’

The STARS framework is a set of practical guidelines to help modellers share their simulation models in a way that makes them:

✅ Accessible – so others can easily find and access your model.

✅ Reusable - so people can use and adapt the model for their own work.

✅ Understandable and well-documented - so others can engage with and understand your model better.

STARS Components

Overview of the STARS Framework from Monks et al. (2024) (CC BY 4.0).

Documentation with Quarto

It’s possible to set up a Quarto project within the same

Cool Finds - New Streamlit Features

Key useful things in 1.43.0

Read the full release notes here

Key useful things in 1.42.0

Read the full release notes here

Key useful things in 1.39.0

Read the full release notes here

Upgrading Streamlit

Make sure you update version numbers in your requirements.txt or environment.yml file!

Cool Finds - Streamlit Icon Metric Cards

I’ve adapted some code to make a neat little metric card that can acept icons from the material icon library.

Code and output

Show the function code 
def iconMetricContainer(key,icon_unicode,css_style=None,icon_color='grey', family="filled", type="icons"):
    """Function that returns a CSS styled container for adding a Material Icon to a Streamlit st.metric value

    CREDIT for starter version of this code: https://discuss.streamlit.io/t/adding-an-icon-to-a-st-metric-easily/59140?u=sammi1

    Args:
        key (str): Unique key for the component
        iconUnicode (str): Code point for a Material Icon, you can find them here https://fonts.google.com/icons. Sample \e8b6
        css_style(str, optional): Additional CSS to apply
        icon_color (str, optional): HTML Hex color value for the icon. Defaults to 'grey'.
        family(str, optional): "filled" or "outline". Only works with type = "icons"
        type(str, optional): "icons" or "symbols"

    Returns:
        DeltaGenerator: A container object. Elements can be added to this container using either the 'with'
        notation or by calling methods directly on the returned object.
    """

    if (family == "filled") and (type=="icons"):
        font_family = "Material Icons"
    elif (family == "outline") and (type=="icons"):
        font_family = "Material Icons Outlined"
    # elif (family == "filled") and (type=="symbols"):
    #     font_family = "Material Symbols"
    elif type=="symbols":
        font_family = "Material Symbols Outlined"
    else:
        print("ERROR - Check Params for iconMetricContainer")
        font_family = "Material Icons"

    css_style_icon=f'''
                    div[data-testid="stMetricValue"]>div::before
                    {{
                        font-family: {font_family};
                        content: "\{icon_unicode}";
                        vertical-align: -20%;
                        color: {icon_color};
                    }}
                    '''

    if css_style is not None:
        css_style_icon += """

        """

        css_style_icon += css_style

    iconMetric=stylable_container(
                key=key,
                css_styles=css_style_icon
            )
    return iconMetric
## EXAMPLE USAGE
my_formatted_value_for_metric="6.3%"

with iconMetricContainer(key="nonattend_metric", icon_unicode="e61f", family="outline"):
    st.metric("Average Number of Calls Resource Couldn't Attend",
            my_formatted_value_for_metric,
            border=True)

Cool Finds - Expandable Code Snippets in VSCode

Is there something clunky you have to keep looking up in VSCode?

Turn it into your own reusable snippet!

Opening the ‘Configure Snippets’ Menu

Open the menu from file –> preferences

Choosing the snippet language

We will now be presented with a list of possible snippet languages.

If working with Quarto, I’d recommend you use ‘markdown’ rather than ‘Quarto’, as then there’s a neat little shortcut in VSCode we can use to access snippets while writing quarto docs (CTRL + SPACE)

Alternatively, select Python

Writing our snippet - Quarto

Here, I’ve written a sample

You can find more out writing snippets here

Show the code 
{
    // Place your snippets for markdown here. Each snippet is defined under a snippet name and has a prefix, body and
    // description. The prefix is what is used to trigger the snippet and the body will be expanded and inserted. Possible variables are:
    // $1, $2 for tab stops, $0 for the final cursor position, and ${1:label}, ${2:another} for placeholders. Placeholders with the
    // same ids are connected.
    // Example:
    // "Print to console": {
    //  "prefix": "log",
    //  "body": [
    //      "console.log('$1');",
    //      "$2"
    //  ],
    //  "description": "Log output to console"
    // }

    "Make a two-column layout": {
        "prefix": "col2",
        "body": [
            ":::: {.columns}",
            "",
            "::: {.column width='50%'}",
            "",
            ":::",
            "",
            "::: {.column width='50%'}",
            "",
            ":::",
            "",
            "::::"
        ],
        "description": "Create a column layout in quarto with two columns of equal width"
    },

    "Make a fragment": {
        "prefix": "frag",
        "body": [
            "::: {.fragment}",
            "",
            ":::"

        ],
        "description": "Create a quarto fragment for incremental revealing of content"
    },

    "Make an incremental list": {
        "prefix": "inc",
        "body": [
            "::: {.incremental}",
            "",
            ":::"

        ],
        "description": "Create a quarto incremental list container for revealing one bullet point at a time"
    }
}

Using our Quarto snippet

Now, when I press CTRL + SPACE, I can type in my snippet IDs and select them to make them populate my document

Writing and using our snippet - Python

Python snippets are similar!

Here we’ve defined a snippet called ‘condlist’ for generating a conditional list comprehension.

Show the code 
{
    // Place your snippets for python here. Each snippet is defined under a snippet name and has a prefix, body and
    // description. The prefix is what is used to trigger the snippet and the body will be expanded and inserted. Possible variables are:
    // $1, $2 for tab stops, $0 for the final cursor position, and ${1:label}, ${2:another} for placeholders. Placeholders with the
    // same ids are connected.
    // Example:
    // "Print to console": {
    //  "prefix": "log",
    //  "body": [
    //      "console.log('$1');",
    //      "$2"
    //  ],
    //  "description": "Log output to console"
    // }

    "Conditional list comprehension": {
        "prefix": "condlist",
        "body": [
            "['Even' if n % 2 == 0 else 'Odd' for n in a]"
        ],
        "description": "Provide an adaptable template for conditional logic in a list comprehension"
    }

}

To find out more on snippets, click here